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Abstract 

Background: Evidence mapping describes the quantity, design and characteristics of research in broad topic areas, 
in contrast to systematic reviews, which usually address narrowly-focused research questions. The breadth of 
evidence mapping helps to identify evidence gaps, and may guide future research efforts. The Global Evidence 
Mapping (GEM) Initiative was established in 2007 to create evidence maps providing an overview of existing 
research in Traumatic Brain Injury (TBI) and Spinal Cord Injury (SCI). 

Methods: The GEM evidence mapping method involved three core tasks: 

1. Setting the boundaries and context of the map: Definitions for the fields of TBI and SCI were clarified, the 
prehospital, acute inhospital and rehabilitation phases of care were delineated and relevant stakeholders (patients, 
carers, clinicians, researchers and policymakers) who could contribute to the mapping were identified. Researchable 
clinical questions were developed through consultation with key stakeholders and a broad literature search. 

2. Searching for and selection of relevant studies: Evidence search and selection involved development of specific 
search strategies, development of inclusion and exclusion criteria, searching of relevant databases and independent 
screening and selection by two researchers. 

3. Reporting on yield and study characteristics: Data extraction was performed at two levels - 'interventions and study 
design' and 'detailed study characteristics'. The evidence map and commentary reflected the depth of data extraction. 

Results: One hundred and twenty-nine researchable clinical questions in TBI and SCI were identified. These 
questions were then prioritised into high (n = 60) and low (n = 69) importance by the stakeholders involved in 
question development. Since 2007, 58 263 abstracts have been screened, 3 731 full text articles have been reviewed 
and 1 644 relevant neurotrauma publications have been mapped, covering fifty-three high priority questions. 

Conclusions: GEM Initiative evidence maps have a broad range of potential end-users including funding agencies, 
researchers and clinicians. Evidence mapping is at least as resource-intensive as systematic reviewing. The GEM 
Initiative has made advancements in evidence mapping, most notably in the area of question development and 
prioritisation. Evidence mapping complements other review methods for describing existing research, informing 
future research efforts, and addressing evidence gaps. 



Background 

All stakeholders in health care like to have research evi- 
dence to inform their decision-making. Some decisions 
relate to questions that are narrowly focused, such as 
the effectiveness of an intervention or the accuracy of a 
diagnostic test. Others are much broader, such as the 
relative effectiveness of various treatment options 
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available for a particular condition. The methods of sys- 
tematic reviews are well developed for the former group, 
but the latter poses many challenges for locating, orga- 
nising, collating and synthesising information into a use- 
ful, understandable format, as it requires an organised 
up-to-date reference source of research evidence addres- 
sing broad content areas. 

For this task, existing evidence resources seem gener- 
ally inadequate. Finding all relevant publications on a 
topic using PubMed-type databases is a long and labor- 
ious task. Systematic reviews are rarely completely up- 
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to-date. Being the product of a largely investigator-dri- 
ven collaboration, the Cochrane Library doesn't necessa- 
rily contain reviews encompassing all questions in a 
given topic area. Furthermore neither searches for pri- 
mary studies, nor systematic reviews are well suited to 
identifying important evidence gaps. 

Techniques for describing existing evidence in a broad 
content area are relatively new, and have been called 
evidence mapping and scoping studies [1,2]. How they 
compare with systematic reviews is shown in Table 1. 
Importantly, evidence mapping and scoping does not 
extend to the quality appraisal and synthesis techniques 
associated with systematic reviews. Scoping is further 
distinguished from mapping by the inclusion of research 
results in the description of relevant evidence. Three 
publications have outlined methods for mapping or 
scoping a content area [3-5], which have included three 
core tasks: 

1. Setting the boundaries and context of the topic 
area in question; 

2. Searching for and selection of relevant studies; 
and 

3. Reporting on yield and study characteristics. 

In 2007 the Global Evidence Mapping (GEM) Initia- 
tive was established with funding from the Victorian 
Neurotrauma Initiative (VNI), a research funding agency 
established by the Victorian Government's Transport 
Accident Commission and Department of Innovation 
Industry and Regional Development [6]. The GEM 
Initiative developed as a collaboration of clinical, 
research and policy stakeholders to provide an overview 
of existing research about Traumatic Brain Injury (TBI) 
and Spinal Cord Injury (SCI). 

The GEM Initiative aimed to map the available 
research for the prehospital, acute inhospital and rehabi- 
litation phases of care for TBI and SCI. The breadth of 



clinical research questions and volume of studies in 
these areas is considerable. The evidence mapping 
approach was thought to be the most appropriate 
method because its main purpose is to provide an over- 
view of a broad range of research questions and identify 
evidence gaps [3-5]. In creating the SCI and TBI evi- 
dence maps, the GEM Initiative drew on and advanced 
evidence mapping methods. Importantly, we developed 
an in-depth question development process, which 
enabled us to satisfy a key project aim - the identifica- 
tion of evidence gaps - by comparing and contrasting 
stakeholder-driven clinical research questions with pub- 
lished literature. 

This article focuses on the GEM Initiative's evidence 
mapping methods and illustrates how these methods 
were applied to neurotrauma (TBI and SCI) topics. The 
results of the mapping process itself are also briefly 
described. 

Methods 

Figure 1 summarises the steps of the GEM Initiative evi- 
dence mapping process (centre of figure), their relation- 
ship to previously published core evidence mapping 
tasks (left of figure) and the resulting outputs (right of 
figure). The figure also illustrates the boundary between 
evidence mapping and the other secondary research 
methods described in Table 1. 

Because evidence mapping outputs have stand-alone 
merit and application, the GEM evidence mapping pro- 
cess, like previously published evidence mapping meth- 
ods, is not an 'all or nothing' proposition; the number of 
steps undertaken and the breadth of the topic area cov- 
ered can therefore vary according to available resources. 
An evidence map can form the basis of a scoping review 
or number of individual systematic reviews, as evidence 
mapping produces source materials for both these activ- 
ities (Table 1). Further, analysis of the evidence gaps 
identified in the mapping process can aid planning of 



Table 1 Methods of identifying and collating research evidence 



Method 



Definition 



Purpose 



Breadth Depth of process 



Systematic "an overview of primary studies which 
Review contains an explicit statement of 

objectives, materials and methods and has 
been conducted according to explicit and 
reproducible methodology." [22] 



Summarise overall quality and 
results of a body of research; 
inform clinical practice 



Addresses a 

focused 

clinical 

question 

[13] 



In depth searching, quality appraisal and 
synthesis of studies relevant to the 
identified clinical question 



Scoping Overview of "the key concepts 
Study underpinning a research area and the 

main sources and types of evidence 
available" Mays et al. 2001; cited in [3] 



Examine the extent, range and Covers a 
nature of research activity; identify broad topic 
research gaps [3] area 



Identifying boundaries and context of 
the area under study, followed by 
searching, collation and summary of 
study characteristics and results with no 
quality appraisal or synthesis 



Evidence The systematic organisation and 
Mapping illustration of a broad field of research 
evidence [3,5] 



Characterise the breadth, depth, Covers a 

methodology of relevant evidence broad topic 

and make this readily accessible area 
[5]; identify research gaps 



Identifying boundaries and context of 
the area under study and providing a 
description of yield, interventions, study 
design and study characteristics 
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future research by reducing duplication of research 
effort within topic areas, and focussing research efforts 
on high priority topics (Figure 1). 

Following is an in-depth description of the GEM 
Initiative evidence mapping methods, illustrated with 
specific examples from our evidence maps. Emphasis 
has been placed upon the unique aspects of these meth- 
ods; the use of a mapping workshop to identify clinical 
research questions, development of questions, question 
prioritisation and the two levels of data extraction and 
presentation. 

Core task 1: Setting the boundaries and context of the 
map 

The GEM Initiative approached this task using two dis- 
tinct steps - question development and question 
prioritisation. 

STEP 1: Question Development 

The question development phase involved four broad 
data collection exercises designed to gain an overview of 



the scope of the field and gather perceptions of key sta- 
keholders - patients, carers, clinicians, researchers and 
policymakers - regarding the key clinical issues in the 
field of interest. The data gathered from these exercises 
was used to develop searchable clinical questions. 
1A: Expert Consultation Consultation with experts has 
been identified as a useful strategy for setting the 
boundaries and context of an evidence map [4,5]. 
Experts identify specific terminology, which aids in 
search strategy development, and outline who has stake- 
holder interests. This information informs subsequent 
stages of the mapping process [4,5]. 

To frame the TBI and SCI evidence maps, the GEM 
Initiative consulted with local and international clinical 
and research leaders in these fields. These consultations 
involved introducing the GEM Initiative and the concept 
of evidence mapping, followed by an unstructured discus- 
sion designed to enhance our understanding of specific 
aspects of TBI and SCI care. For example, our discussion 
with experts in SCI rehabilitation encompassed common 



Core Task 1 
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& Context 
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Question Development 

A. Expert consultation 

B. Preliminary literature search 

C. Mapping workshop 

D. Online survey 

E. Development of questions 



Researchable Questions 
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Search & Selection 



STEP 2 

Question Prioritisation 



Prioritised Questions 




STEP 3 

Evidence Search and Selection 

A. Development of search strategy 

B. Development of inclusion criteria 

C. Literature searching 

D. Preliminary screening: titles / abstracts 

E. Full text article procurement 

F. Final screening: full text articles 



Relevant Research 



Core Task 3 
Reporting 




STEP 4 



A. Interventions & study design 

B. Detailed study characteristics 
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Study results 



Systematic Review 
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Evidence Gaps Analysis 

Planning of future research 



Figure 1 GEM Initiative Evidence Mapping Methods. 
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problems experienced by SCI patients in this phase of 
care and the healthcare disciplines involved in managing 
these. 

Although the GEM Initiative focused on clinicians and 
researchers, the other key stakeholders with expert 
knowledge in this area - patients, carers and policy- 
makers (who were involved in later stages in the GEM 
process) - can add different perspectives to this process 
and could also be involved at this stage. 
IB: Preliminary literature search To identify existing 
relevant reviews and guidelines relating to TBI and SCI 
in each of the phases of care, a preliminary literature 
search was conducted by a GEM Initiative project officer 
who had extensive Cochrane systematic review experi- 
ence. Major medical databases were searched using rele- 
vant Medical Index Subject Headings (MeSH) and other 
terms based upon the clinical knowledge of the GEM 
researchers. The focus of the search was on reviews and 
guidelines rather than primary studies. Relevant reviews 
and guidelines were used to identify clinical questions, 
describe the extent to which they had been addressed by 
existing literature and identify additional key literature 
search terms. This information further developed the 
scope of the map. 

1C: Mapping Workshop To ensure the evidence maps 
were comprehensive and useful, key stakeholders 
involved in all phases of TBI and SCI patient care - 
patients, carers, clinicians, researchers and policymakers 
- were invited to participate in mapping workshops. 
This is an enhancement on the techniques for obtaining 
expert opinion described by Katz et al. [5]. 

The primary aim of the mapping workshops was to 
gather stakeholder perspectives on key clinical questions, 
outcomes, patient factors and contextual factors affect- 
ing TBI and SCI care. A total of four mapping work- 
shops were conducted; one each for the pre-hospital 
and rehabilitation phase of care in which both SCI and 
TBI stakeholders were involved; and separate TBI and 
SCI workshops dealing with the acute inhospital phase 
of care. 

The nominal group technique [7] was used. This 
involved individual, anonymous generation of ideas fol- 
lowed by group discussion. The nominal group techni- 
que overcomes the inhibiting influences that can 
operate when group interaction is used to generate ideas 
[7]; examples of such inhibiting influences include a 
health professional deferring to a senior colleague rather 
than offering their own opinion, or a patient not want- 
ing to highlight issues that could be construed as criti- 
cism of their clinicians. 

A professional facilitator with experience in the nom- 
inal group technique ran the workshops. The mapping 
workshops varied in length from one to two hours 
according to the breadth of the field and participant 



availability. A consistent workshop format was employed 
comprising: 

• Introduction: Aims, key definitions, confidentiality/ 
consent procedures. 

• Brainstorming: Writing of ideas on individual 
adhesive notes, without discussion. The PICO format 
('Problem', 'Intervention', 'Comparator', 'Outcome') 
was suggested but not mandatory. Clinical issues 
could include diagnosis, prognosis, interventions, 
and service delivery/organisation. The adhesive notes 
were then placed on large sheets around the room. 

• Prioritisation: During a short refreshment break, 
GEM Initiative researchers grouped similar notes. 
Participants then prioritised the notes by placing a 
limited number of adhesive dots next to the notes 
they perceived as most important. 

• Discussion: Open discussion was conducted on the 
ideas generated, the results of the prioritisation and 
any further issues identified. This was a valuable way 
of gaining insight into other perspectives and elabor- 
ating, modifying and developing consensus [7]. 
Where time permitted, the issue of contextual fac- 
tors (for example rural or urban location, socioeco- 
nomic status) and their influence on the feasibility of 
delivering an intervention, the intervention itself, 
outcomes, or other aspects of care was discussed. 

• Conclusion: Participants were thanked and invited 
to participate in subsequent question development 
and prioritisation activities and/or receive a copy of 
the final report. 

The mapping workshops resulted in an unstructured 
collection of ideas and PICO terms of interest as well as 
a sense of the relative importance of these ideas. 
ID: Online survey An online survey was used to aug- 
ment the information gathered in the mapping work- 
shops. Only one published evidence mapping method has 
incorporated the use of a survey. The GEM Initiative evi- 
dence mapping survey built on the method described by 
Katz et al. [5] by placing an emphasis on the PICO ele- 
ments that form the basis of searchable clinical questions 
and, like the mapping workshop, bringing patients into 
the process of generating ideas and questions. 

Survey recipients were identified through GEM clinical 
and research contacts, primary searching of hospital and 
academic websites and organisational mailing lists, 
including a mailing list of clinicians and researchers on 
the VNI database to which GEM staff were blinded. The 
survey comprised three sections: 

1. Introduction and background: Brief background to 
the project, purpose, confidentiality and contact 
information. 
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2. Survey of PICO terms of interest: Participants were 
asked to fill in a minimum of two blank PICO tem- 
plates identifying a series of clinical problems, thera- 
pies and/or outcomes that they considered 
important in their specialty area. 

3. Demographic details: Profession/patient group, 
years experience or years a patient, extent of 
research/clinical involvement in TBI/SCI, geographi- 
cal location, gender. 

The survey data were more closely aligned to the 
PICO format than the mapping workshop data, as the 
survey responses were mandated under PICO headings. 
IE: Development of questions These four activities (1A 
-ID) generated a very large volume of unstructured 
ideas including single words, PICO fragments, state- 
ments and questions. They varied in format, language, 
perspective and emphasis because a range of stake- 
holders was involved in the process. 

Organisation of this heterogeneous data was required 
in order to promulgate a key aim of the GEM Initiative 
- the identification of evidence gaps - by searching for 
literature pertaining to the identified issues. We 
addressed this need by transforming the data into a list 
of 'answerable' clinical research questions. An 'answer- 
able' question seeks specific knowledge, is framed to 
facilitate literature searching and therefore, follows a 
semi-standardised structure [8,9]. The challenge of 
representing broad stakeholder perspectives in answer- 
able question form has been recognised by Bates et al. 
[4]. The GEM Initiative used an in-depth, systematic 
process for transforming the data from steps 1A - ID 
into answerable clinical research questions that 
involved: 

1. Entering data into an electronic database (Micro- 
soft Excel). 

2. Coding the clinical problems described using 
codes from the International Classification of Func- 
tioning, Disability and Health (ICF) [10,11]. This 
comprehensive framework was suitable for identify- 
ing the broad range of clinical issues across multiple 
phases of TBI and SCI care; for other evidence map- 
ping topic areas, other taxonomies may be more 
appropriate for coding purposes. 

3. Collating coding information to identify the 
range and frequency of all codes represented in the 
data. 

4. Conducting a series of half-day question construc- 
tion meetings in which the GEM Initiative research 
team transformed the data into a list of answerable 
clinical research questions encompassing all ICF 
codes represented in the data. 



The example in Table 2 illustrates the transformation 
of data from steps 1A - ID into an answerable clinical 
research question using this approach. 

A range of question development strategies have been 
reported in published evidence mapping projects. Arksey 
[12] used qualitative analysis to identify the key topics 
and issues arising from stakeholder consultation in her 
scoping review of interventions for carers of people with 
mental health problems. Bates et al. [4] used a PICO- 
style approach to frame inclusion and exclusion criteria 
for literature search and other mapping tasks. Similarly, 
Arksey et al. [3] discussed the importance of defining key 
question parameters and recommended framing such 
parameters to maximise breadth of literature coverage. 

The GEM Initiative approach to question development 
built upon these strategies by using the ICF framework 
to organise large amounts of unstructured data and 
detailing the specific tasks involved in transforming this 
data into answerable clinical research questions. This 
approach facilitated the development of a large number 
(129) of answerable research questions across an entire 
research field. The comprehensive question development 
approach used by the GEM Initiative ensured that all of 
the ideas identified by the key stakeholders in steps 1A - 
ID - patients, carers, clinicians, researchers and policy- 
makers - were represented in the final list of answerable 
clinical research questions for each phase. 
STEP 2: Question Prioritisation 

Bates et al. [4] recognised that the evidence mapping 
process involves calibrating the answerable questions 
developed against the resources available to map these 
questions. Given the number of questions identified in 
the GEM Initiative question development process, there 
was a clear need to prioritise these as our project 
resources did not enable all steps of the mapping pro- 
cess to be applied to all identified questions. 

The GEM Initiative involved the key stakeholders 
from the question development process in prioritising 
the list of clinical research questions from Step 1. An 
online survey of patients, clinicians and researchers was 
conducted. Survey recipients included participants from 
the mapping workshop, the GEM advisory group and 
other representatives with expert knowledge of TBI and 
SCI. Survey respondents were asked to rank each 
searchable clinical question on three domains - clinical 
importance ('How clinically important is this question?'), 
novelty ('Does this question represent an emerging area 
of clinical practice?') and controversy ('What is the level 
of variability regarding opinion and/or practice for this 
question?') - using a scale of 1 to 4, where: 

1 = not at all 

2 = a little 
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Table 2 Transformation of question development data into an answerable clinical research question 



Data source 


Example of data 


ICF code 
[10,11] 


Resulting answerable clinical research 
question 


1A: Expert 
consultation 


"Spasticity" 


b735: Muscle 
tone 

functions 




1B: Preliminary 

iterature 

search 


"What are the best pharmacological treatments to manage TBI 
spasticity?" 

"How effective is intrathecal baclofen, botulinum toxin type A, passive 
exercises etc?" 


b735: Muscle 
tone 

functions 


What are effective interventions for 
preventing and managing skeletal muscle 
spasticity? 


1C: Mapping 
workshop 


"Problem: spasticity management" 

"Decreased joint range of motion and integrity" 

"Spasticity causing functioning or safety problem" 


b735: Muscle 

tone 

functions 




1D: Online 
survey 


"P: Spasticity 

I: Pharmacological management of tonal changes, surgical 
intervention, positioning techniques, gentle movement program, 
patient/family education 

0: Maximise passive and/or active joint range and maintain optimal 
joint integrity" 


b735: Muscle 
tone 

functions 




ICF: International Classification of Functioning, Disability and Health. 



3 = a moderate amount 

4 = high. 

The questions were then ranked into two broad cate- 
gories: 

♦ High priority questions: Questions where the 
majority of respondents ranked 'Clinical Importance' 
as high AND ('Novelty' OR 'Controversy' as high or 
moderate). 

♦ Low priority questions: All other questions. 

This ranking procedure was arbitrary and driven by 
the imperatives of the GEM Initiative project; other 
research groups may use other ranking parameters and 
divisions consistent with the focus of their research 
activity. For example, cost-effectiveness or feasibility of 
undertaking primary or secondary research, rather than 
clinical importance, may be deemed the most important 
determinants of which questions to pursue. 

Core task 2: Searching for and selection of relevant 
studies 

STEP 3: Evidence Search and Selection 

The evidence search and selection methods used by the 
GEM Initiative were consistent with accepted principles 
of systematic reviewing. Specifically, these were: 

1. In-depth search strategies using applicable MeSH 
headings and keywords [9,13]. 

2. Pre-determined inclusion criteria [4,13]. 

3. Searching a broad range of medical databases. 

4. Review of retrieved citations/full-text articles 
against the question-specific inclusion criteria by 
two independent reviewers to reduce bias [3,13]. 



As such methods are extensively documented in sys- 
tematic review methodology literature, only a brief 
description of their specific application to the GEM 
Initiative follows. Examples are contained in Appendix 1. 
3A: Development of search strategy An information 
specialist was used to develop and run searches and col- 
late search results. Search strategies were built by com- 
bining at least two of: 

• Core terms representing TBI and SCI 

• Filter terms, for example to target rehabilitation 
literature 

• Intervention terms specific to the clinical questions 
being mapped. 

GEM Initiative researchers with a clinical background 
(the research team included two medical doctors, a phy- 
siotherapist and a nurse) contributed their clinical 
knowledge of the subject areas to search strategy devel- 
opment. Abridged versions of the search strategy were 
used for the databases that had a simpler search inter- 
face or smaller content, such as non-English databases, 
as these databases could not accommodate a large num- 
ber of search terms [4,14]. 

3B: Development of inclusion criteria A priori eligibil- 
ity criteria were developed regarding study type (sys- 
tematic reviews and a broad range of primary study 
designs encompassing therapy, diagnosis and prognosis 
were included; animal, laboratory, cadaver and simula- 
tion studies were excluded), participants (explicit defini- 
tions of TBI and SCI) and phase of care (prehospital, 
acute inhospital, rehabilitation; injury prevention 
research was excluded). 

Given the breadth of topics covered, question-specific 
refinements to the inclusion criteria were sometimes 
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required after preliminary database searching and 
review; for example, to determine which clinical sub- 
categories of a particular disorder were most relevant. 
Such post hoc refinements, which can only occur when 
some literature has been examined [3], also aided in the 
classification of eligible studies. For example, the man- 
agement of spasticity in TBI could be divided into surgi- 
cal, pharmacological and physical therapies. Where 
necessary, changes to the search strategy were made to 
reflect these alterations and refinements. A similar itera- 
tive approach has been adopted by others [4]. 
3C: Literature searching Major medical databases 
(including non-English) and trials registries were 
searched for all phases of care covered by the GEM 
Initiative. Searching of further databases tailored to the 
specific needs of each phase/question was also con- 
ducted. For example, for the question pertaining to 
spasticity, three additional databases were searched to 
reflect the multidisciplinary nature of this intervention 
in addition to the databases routinely searched (Appen- 
dix 1). 

For all searches there was no restriction on year of 
publication or language. For each question all of the 
chosen databases were searched on the same day and a 
reference library (Endnote) of search results by database 
was compiled. The reference library was then de-dupli- 
cated for the purpose of screening and selection. 
3D: Preliminary screening: titles/abstracts Retrieved 
citations were reviewed against the question-specific 
inclusion criteria by two independent reviewers. Where 
necessary, a third reviewer was used to resolve disagree- 
ments by breaking a tie. 

3E: Full-text article procurement Full-text articles 
identified during screening were procured either directly 
or via order from university/hospital libraries. 
3F: Final screening: full-text articles Full-text articles 
were reviewed against inclusion criteria as described 
above (3D). 

Records of screening and selection decisions (citations 
excluded; citations identified for full text review; cita- 
tions agreed upon for full text review; studies excluded 
upon full-text review; studies identified for inclusion in 
evidence map; studies agreed upon for inclusion in evi- 
dence map) were systematically kept. 

As a search of all the major and auxiliary databases 
may inadequately reflect the total relevant research on 
any topic [15,16], reference lists of relevant articles were 
systematically searched by GEM reviewers [13] and 
expert clinicians were consulted to identify further 
potentially relevant articles. 

Core task 3: Reporting on yield and study characteristics 

The GEM Initiative undertook two levels of evidence 
mapping reporting; an overview of interventions (where 



relevant) and study design of included studies and a 
more in-depth exploration of study characteristics. In 
each case, the output tables and text identified and 
articulated evidence gaps by describing how the relevant 
literature addressed the research question. These levels 
of evidence mapping are described and illustrated below 
with examples from the GEM Initiative evidence maps. 
STEP 4: Data Extraction 

A relational data extraction database was created 
(Microsoft Access), which could be interrogated to 
explore and query the evidence map [4]. 

For the prehospital evidence map, an in-depth data 
extraction and review of study quality was undertaken. 
However this process proved to be unfeasible once the 
volume and breadth of literature in subsequent phases 
became apparent. Furthermore, it was felt that quality 
appraisal, summary of findings and specific outcomes of 
individual studies was better conducted within a formal 
systematic review, and we have distinguished evidence 
maps and systematic reviews in part on this level of data 
extraction and reporting as outlined earlier. This 
resulted in the development of the two data extraction 
protocols that were used in the rehabilitation and 
acute inhospital evidence maps: 

4A: Interventions and study design Extracting to this 
level involved identifying the number of studies by study 
design for each intervention addressing the question. 
Where the question did not pertain to an intervention, 
relevant studies were listed by study design only. For all 
phases of the project, study design was classified accord- 
ing to the definitions outlined in the Australian National 
Health and Medical Research Council (NHMRC) levels 
of evidence [17] and Straus et al. [9]. A brief commen- 
tary on evidence summarised the key findings regarding 
the yield and breakdown of study interventions and 
designs (Table 3). 

4B: Detailed study characteristics The following data 
were extracted: study design, country, sample size, 
source population, interventions, outcome measures, 
patient factors (demographics, injury classification). This 
facilitated a more in-depth evidence map for the clinical 
question, as reflected by both the output table and the 
scope of the commentary on evidence (Table 4). 

Use of these two data extraction methods facilitated a 
broad coverage of clinical questions in these areas 
within project resources. The decision regarding which 
data extraction method to use involved consideration of 
a range of factors including the nature and complexity 
of the question and the need to balance consistency, 
comprehensiveness and question coverage across both 
TBI and SCI topics. 

Complete versions of tables and reports exemplifying 
both styles of data extraction are available at http:// 
www.evidencemap.org. This site contains a searchable 
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Table 3 Example of 'interventions and study design' output: Prevention and management of skeletal muscle spasticity 
in the rehabilitation phase of TBI 

Intervention n SR RCT X-over Cohort ITS Case series Case report 

Casting/splinting 16 1 2 12 5 5 

TENS 2 1 1 

Baclofen: Intrathecal long term 16 1 3 11 1 

Baclofen: Intrathecal bolus/test dose 5 1 3 1 

Baclofen: Oral 1 1 

Botulinum toxin (Botox) 10 3 12 4 

Clonidine 1 1 

Lower leg casting/splinting & Botox 1 1 

Cryotherapy 1 1 

Phenol nerve block 1 1 

Weight-bearing gait retraining 1 1 

Divolproex sodium 1 1 

Bobath treatment 1 1 

Voice and respiration treatment 1 1 

Rhythmic, passive movement 1 1 

Seating 1 1 

Combination of therapies 1 1 

SR: Systematic Review. 

RCT: Randomised Controlled Trial. 

X-over: Crossover trial. 

ITS: Interrupted Time Series. 

TENS: Transcutaneous Electrical Nerve Stimulation. 

Commentary on evidence. 

Over half of the 61 studies investigated pharmacological/neurotoxin therapies for managing spasticity in TBI. However, no systematic review of this literature was 
identified. The next most common intervention under investigation was casting/splinting. One systematic review of this intervention was identified [23]. Only ten 
of the 61 studies investigated therapies other than pharmacological/neurotoxin agents or casting/splinting. 



database of all TBI and SCI topics for which an evidence 
map has been completed. For each topic, either the out- 
put tables ('maps') alone (as illustrated in Tables 3 and 
4) can be downloaded, or a report is available containing 
the maps plus a background to the topic, the inclusion 
criteria for search, the search strategy and yield, a 
description of the map, a commentary on evidence and 
a reference list. The date of last update is also listed for 
each topic. 

Results 

Although this paper focuses on the steps of the evidence 
mapping process, a brief overview of the findings of the 
GEM mapping process itself illustrates its usefulness in 
identifying evidence gaps. 

Question development 

Since the inception of the GEM Initiative, a total of 129 
answerable clinical research questions have been gener- 
ated, covering the prehospital (n = 30; 17 TBI, 13 SCI), 
acute inhospital (n = 37; 18 TBI, 19 SCI) and rehabilita- 
tion (n = 62; 26 TBI, 36 SCI) phases of TBI and SCI 
care. 



Question Prioritisation 

The prioritisation procedure undertaken in the GEM 
Initiative resulted in 60 of the 129 clinical research 
questions being identified as high priority; 30 in each of 
TBI and SCI. 

Evidence Search and Selection 

Since 2007, 58 263 abstracts have been screened, 3 731 
full text articles have been reviewed and 1 644 relevant 
neurotrauma publications have been identified. 

Data Extraction 

Data extraction has been completed to produce evidence 
maps for fifty- three high priority questions. For 11 of 
the 53 questions (3 TBI, 8 SCI), gaps in primary 
research evidence were identified: 

♦ For six questions (2 TBI, 4 SCI) no studies were 
identified. 

♦ For a further four questions pertaining to interven- 
tions (1 TBI, 3 SCI), only case series, case study or 
cross-sectional study designs were identified. These 
represent neurotrauma research topics in which 
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Table 4 Example of 'detailed study characteristics' output (extract only): Effective interventions for optimising bowel 
function in the rehabilitation phase of SCI 

Reference Study Country n Pop. Patient Condition Intervention Outcomes 
Design Group 

Ayas et al. Case series Turkey 24 All Adult Neurogenic Abdominal Mean time for bowel evacuation; Frequency of 

2006 [24] SCI bowel massage defecation; Faecal incontinence; Abdominal 

distension; Abdominal pain; Difficult intestinal 
evacuation 

Furusawa Case series Japan 15 All Adult Autonomic Bowel program BP; Pulse; Symptoms of cervical SCI 

et al. 2007 SCI Dysreflexia involving manual 

[25] removal of stool 

Luther et Retrospective USA 370 All Adult Neurogenic Bowel care Training for bowel care program; Quality of life; 

al. 2005 cohort SCI bowel program; Subjective complication rates 

[26] Colostomy 

Commentary on evidence. 

Two systematic reviews were identified in this topic area covering colostomy and ileostomy and multimodal bowel management and over half (21) of the 
eligible primary studies were case series. Five studies recruited over 100 patients. Four of these were exclusively SCI populations, however, the largest study in 
the evidence map that was on multimodal bowel management (n = 837), comprised a mixed population with no subgroup analysis. The majority of studies 
assessed specific physiological parameters, such as time to flatus and number of bowel evacuations. Only seven studies assessed quality of life measures. 
(Note: this commentary refers to studies not contained in the above table, which is an extract of a table containing the 34 included studies). 



comparative studies (for example, Randomised Con- 
trolled Trial, Pseudo-Randomised Controlled Trial, 
Case-Control Study) are needed, as reflected by the 
NHMRC levels of evidence for intervention studies 
[17]. 

♦ For a further question pertaining to prognosis in 
SCI, no prospective cohort studies were identified, 
which are the highest ranked primary study for 
investigating prognosis [17]. 

TBI topics with primary evidence gaps included pre- 
hospital therapeutic hypothermia and participation in 
leisure and social activities. SCI topics with primary evi- 
dence gaps included prehospital antiemetic medications 
and interventions for sleep disordered breathing. 

A further 12 questions (9 TBI, 3 SCI) contained sec- 
ondary evidence gaps; more than one comparative study 
were identified, but no systematic review. These are 
neurotrauma research topics in which research synthesis 
may be warranted. TBI topics with secondary evidence 
gaps included prehospital hyperventilation and interven- 
tions for minimising drug and alcohol misuse. SCI 
topics with secondary evidence gaps included interven- 
tions for atelectasis/chest infections and interventions 
for optimising nutrition and body weight. 

Discussion 

The GEM Initiative has produced evidence maps that 
may be useful to research funders, researchers, clinicians 
and other research consumers. An extensive list of 
prioritised research questions in a topic area, even in 
the absence of study retrieval and data extraction, is a 
potential springboard for research, research funding and 
policy development. Because the question development 
and prioritisation process is driven by consultation with 
key stakeholders - patients, carers, clinicians, researchers 



and policymakers - this list is more likely to be relevant 
to real-world issues than research questions driven 
solely by academic researchers. Our experience of the 
power of engaging with a wide stakeholder group is 
shared by Arksey et al. [3], who reflected on the addi- 
tional insights that this approach gives to the process. 
Developing and prioritising research questions by con- 
sulting with a number of stakeholder groups carries the 
dual advantage of adding breadth to the research topics 
identified and diluting the potential influence of indivi- 
dual or stakeholder group biases on the question devel- 
opment process. 

Funding agencies can use evidence maps to access 
completed/ongoing studies in a topic area; for example, 
studies using intrathecal baclofen for spasticity in TBI. 
This information can aid in evaluating the need for 
further funded research when reviewing a grant applica- 
tion. The collection of evidence maps across a whole 
topic area (for example, TBI rehabilitation) also high- 
lights areas of high and low research activity, potentially 
informing strategic research funding decisions. The VNI 
exemplified this application of evidence maps through 
its explicit instructions to funding applicants to consult 
the GEM evidence maps in the process of preparing 
their applications [18]. Other research-informed agen- 
cies could also benefit from evidence maps - for exam- 
ple, a policy-maker could gain an understanding of the 
volume and nature of research in a topic area to aid in 
funding or other policy decisions. 

Researchers can use evidence maps to inform research 
decisions and designs. Evidence maps parallel all steps 
of a traditional systematic review except in-depth quality 
appraisal and synthesis. Systematic reviewers can there- 
fore use evidence maps as a resource for updating an 
existing systematic review, or to evaluate the feasibility 
and resource requirements of a systematic review based 
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upon the search yield, and proceed to full study apprai- 
sal and synthesis using this resource. One example of 
such a review is a Cochrane review of interventions for 
managing skeletal muscle spasticity, which built upon 
the GEM evidence map of this topic [19]. As illustrated 
in the results, an evidence map of a specific topic can 
also avoid duplication of primary research and deter- 
mine where further primary studies are needed or con- 
versely, identify if a systematic review of existing studies 
may be more appropriate. Studies included in an evi- 
dence map could also aid primary research planning, for 
example by describing outcome measures or research 
protocols. 

Clinicians can use evidence maps to rapidly access 
research information pertaining to specific populations 
and interventions. For example, a case manager could 
access literature about the effectiveness of community 
integration programs for TBI specifically in paediatric 
populations, or focusing on computer-based interven- 
tions. However, it should be emphasised that evidence 
mapping does not purport to guide clinical practice, as 
the studies in the GEM Initiative evidence maps have 
not been subject to quality appraisal or had their results 
synthesised. 

This limitation of evidence mapping necessitates cau- 
tion especially in cases where a large volume of evidence 
is identified for a topic. For example, for eleven of the 
GEM topics (4 TBI, 7 SCI) over 50 relevant English arti- 
cles were identified. These represent neurotrauma 
research topics in which knowledge translation research, 
rather than more primary or secondary studies, may be 
appropriate. However, this cannot be surmised from the 
yield alone, but by combining this with an examination 
of the quality of the comparative studies and/or sys- 
tematic reviews. Should the research be of high quality 
and consistency, knowledge translation research may be 
warranted. If this is not the case, further primary or sec- 
ondary studies are required - just as in cases where 
there is a low volume of evidence. Avoiding the poten- 
tial hazard of misreading evidence maps is best facili- 
tated by transparency in the description of the purpose 
and methods of evidence mapping, in particular by 
drawing a clear distinction between evidence mapping 
and systematic reviewing (Figure 1); ensuring that the 
results of the evidence maps themselves are not pre- 
sented as definitive statements on the direction of 
research evidence; and promoting research that fills the 
evidence gaps identified by the mapping process. 

Identification of and prolonged engagement with the 
likely end-users (clinicians, researchers, funding agen- 
cies) starting early in an evidence mapping project 
enables tailoring of evidence mapping activities and out- 
puts to their priorities. Although there was provision in 
the evidence mapping process for the identification and 



mapping of non-intervention questions, the existing 
GEM evidence resource predominantly contains maps 
pertaining to interventions. This could be in part due to 
the use of PICO to guide question development; it may 
be a reflection of the priorities of those involved in 
question development; or it could be a simple illustra- 
tion that most clinical research questions are interven- 
tion-focused. In this context, some tailoring to the 
evidence mapping process could be necessitated (for 
example, less emphasis on PICO) if outputs focused on 
non-intervention issues are identified as important dur- 
ing initial consultation with end-users. 

There are many resource-intensive steps in evidence 
mapping. As described, the GEM Initiative evidence 
maps involved the generation of 129 clinical questions 
and independent review of tens of thousands of 
abstracts and thousands of full-text articles by the 3.5 
EFT project staff. A similar level of staffing has been 
reported by Arksey et al. [3]. Preliminary calculations 
performed by the GEM Initiative, based on time spent 
to develop detailed search strategies, search databases 
and select studies, indicate a resource cost of 1.5 min- 
utes for each citation reviewed [20], plus the costs of 
setting the boundaries and context of the map and 
reporting on yield and study characteristics. 

Our experiences in this project combined with the 
available cost data reinforce the statement by Arksey et 
al. [3] that "the scoping study should not be seen as a 
cheap alternative to the systematic review" (p. 29). Pri- 
marily this is because any savings in resource use from 
not synthesising and appraising studies to the depth of a 
systematic review are easily overtaken by the costs asso- 
ciated with mapping a much broader topic area. Given 
the resource requirements of evidence mapping, efficien- 
cies (that do not compromise scientific rigor) must be 
explored in order to ensure the long-term sustainability 
of evidence mapping. 

Conclusions 

Since its inception in August 2007, the GEM Initiative 
has advanced the field of evidence mapping. The in- 
depth question development and prioritisation methods 
- in particular the use of mapping workshops to engage 
a broad range of key stakeholders - result in evidence 
maps that are more likely to be relevant to patients, 
carers, clinicians, researchers and policymakers. This 
question development method could easily adapt to any 
situation requiring the generation of clinical questions. 

The study search and selection methods parallel 
accepted systematic review approaches, creating poten- 
tial synergies with systematic reviewing. The possibility 
of fruitful collaborations between evidence mappers and 
systematic reviewers has been identified by other evi- 
dence mapping authors [4,5]. Given the resource- 
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intensity of all secondary research methodologies, the 
strategic advantages and efficiencies of such collabora- 
tions are clear. 

Two levels of data extraction have been explored by 
the GEM Initiative; an overview of interventions and 
study design, and a more in-depth description of study 
characteristics. These highlight the many potential out- 
puts and applications of evidence maps and the ability 
of the evidence mapping process to be tailored to spe- 
cific needs. In highlighting the usefulness of evidence 
maps in scoping research in broad topic areas, it is 
acknowledged that this breadth of coverage comes at 
the cost of evidence synthesis and quality appraisal, 
and therefore evidence maps cannot be used to guide 
clinical practice. This limitation should be borne in 
mind when reporting on and disseminating evidence 
maps. 

Evidence maps complement other forms of reviews for 
use by researchers, clinicians, patients, carers and policy- 
makers with an interest in accessing and using evidence 
resources. 

Appendix 1 - Examples of search and selection 
procedures used by the GEM Initiative 

Step 3A: Search Strategy: Intervention Search string for 
'spasticity' used in Medline (April 2008) 

1. Muscle Spasticity/ 

2. spastkf.mp. 

3. 1 or 2 

4. Botulinum Toxin Type A/ 

5. exp GABA Agonists/ 

6. exp gamma- Aminobutyric Acid/ 

7. "gamma amino butyric acid*".mp. 

8. or/4-7 

9. "Range of Motion, Articular"/ 

10. stretcrr.mp. 

11. Splints/ 

12. position*. mp. 

13. exp Physical Therapy Modalities/ 

14. physiotherap\mp. 

15. Occupational Therapy/ 

16. or/9-15 

17. Rhizotomy/or rhizotomy.mp. 

18. myelotomy.mp. 

19. Cordotomy/or cordotomy.mp. 

20. cordectomy.mp. 

21. or/17-20 

22. or/8,16,21 

23. 3 and 22 

Step 3B: Inclusion criterion: 'Rehabilitation' phase of care 

Subacute care and long term care through community 
management. Subacute care is defined as medical and 



skilled nursing services provided to patients who are not 
in an acute phase of an illness but who require a level 
of care higher than that provided in a long term care 
setting. Long term care includes community manage- 
ment and is defined as extended care for chronic condi- 
tions or disabilities requiring periodic, intermittent, or 
continuous care [21]. 

Step 3C: Databases searched for TBI rehabilitation 
question: What are effective interventions for preventing 
and managing skeletal muscle spasticity? 

Major medical databases: 

Cumulative Index to Nursing and Allied Health Lit- 
erature (CINAHL), OVID Embase, OVID Medline, Psy- 
cINFO, Cochrane Library, ISI Web of Science, PubMed 

Non-English databases: 

India Med, Korea Med, LILACS, Panteleimon 

Trials registries: 

WHO - International Clinical Trials Registry Platform 
(ICTRP), The UK National Research Register 

Further databases (to reflect the multidisciplinary nat- 
ure of this intervention): 

Physiotherapy Evidence Database (PEDro), OTSeeker, 
PsycBITE 

Steps 3D - 3F: Search and selection results for TBI 
rehabilitation question: What are effective interventions 
for preventing and managing skeletal muscle spasticity? 

673 titles/abstracts screened 
152 full- text articles ordered and reviewed 
61 articles included in evidence map (one ongoing 

trial and three non-English studies were also identified) 
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